{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# 对用户进行聚类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据来源于 Kaggle 竞赛: Event Recommendation Engine Challenge  \n",
    "根据 events they’ve responded to in the past\n",
    "user demographic information\n",
    "what events they’ve seen and clicked on in our app\n",
    "推断用户对某个事件是否感兴趣\n",
    "由于用户众多(3w+), 可以对用户进行聚类\n",
    "事件描述信息在 users.csv 文件: 共 110 维特征  \n",
    "user_id: 用户 id  \n",
    "locale: 地区, 语言  \n",
    "birthyear: 出身年  \n",
    "gender: 性别  \n",
    "joinedAt: 用户加入 APP 的时间, ISO-8601 UTC time  \n",
    "location: 地点  \n",
    "timezone: 时区\n",
    "\n",
    "作业要求: \n",
    "根据用户的属性进行聚类(KMeans 聚类)\n",
    "尝试 K=20, 40, 80, 并计算各自 CH_scores.\n",
    "\n",
    "提示: 由于样本数目较多, 建议使用 MiniBatchKMeans.\n",
    "\n",
    "标准:\n",
    "1. 特征工程\n",
    "2. 聚类\n",
    "3. CH_Score 计算\n",
    "4. 画图\n",
    "5. 保存结果到文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 导入工具包\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from sklearn.cluster import MiniBatchKMeans\n",
    "\n",
    "from sklearn import metrics\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取并探索数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>locale</th>\n",
       "      <th>birthyear</th>\n",
       "      <th>gender</th>\n",
       "      <th>joinedAt</th>\n",
       "      <th>location</th>\n",
       "      <th>timezone</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3197468391</td>\n",
       "      <td>id_ID</td>\n",
       "      <td>1993</td>\n",
       "      <td>male</td>\n",
       "      <td>2012-10-02T06:40:55.524Z</td>\n",
       "      <td>Medan  Indonesia</td>\n",
       "      <td>480.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3537982273</td>\n",
       "      <td>id_ID</td>\n",
       "      <td>1992</td>\n",
       "      <td>male</td>\n",
       "      <td>2012-09-29T18:03:12.111Z</td>\n",
       "      <td>Medan  Indonesia</td>\n",
       "      <td>420.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>823183725</td>\n",
       "      <td>en_US</td>\n",
       "      <td>1975</td>\n",
       "      <td>male</td>\n",
       "      <td>2012-10-06T03:14:07.149Z</td>\n",
       "      <td>Stratford  Ontario</td>\n",
       "      <td>-240.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1872223848</td>\n",
       "      <td>en_US</td>\n",
       "      <td>1991</td>\n",
       "      <td>female</td>\n",
       "      <td>2012-11-04T08:59:43.783Z</td>\n",
       "      <td>Tehran  Iran</td>\n",
       "      <td>210.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3429017717</td>\n",
       "      <td>id_ID</td>\n",
       "      <td>1995</td>\n",
       "      <td>female</td>\n",
       "      <td>2012-09-10T16:06:53.132Z</td>\n",
       "      <td>NaN</td>\n",
       "      <td>420.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      user_id locale birthyear  gender                  joinedAt  \\\n",
       "0  3197468391  id_ID      1993    male  2012-10-02T06:40:55.524Z   \n",
       "1  3537982273  id_ID      1992    male  2012-09-29T18:03:12.111Z   \n",
       "2   823183725  en_US      1975    male  2012-10-06T03:14:07.149Z   \n",
       "3  1872223848  en_US      1991  female  2012-11-04T08:59:43.783Z   \n",
       "4  3429017717  id_ID      1995  female  2012-09-10T16:06:53.132Z   \n",
       "\n",
       "             location  timezone  \n",
       "0    Medan  Indonesia     480.0  \n",
       "1    Medan  Indonesia     420.0  \n",
       "2  Stratford  Ontario    -240.0  \n",
       "3        Tehran  Iran     210.0  \n",
       "4                 NaN     420.0  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 读取数据\n",
    "df = pd.read_csv('users.csv')\n",
    "# 查看数据概貌\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(38209, 7)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 查看数据维度\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "共计 38209 个样本, 共 7 维特征. 其中 user_id 不作为聚类属性."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 38209 entries, 0 to 38208\n",
      "Data columns (total 7 columns):\n",
      "user_id      38209 non-null int64\n",
      "locale       38209 non-null object\n",
      "birthyear    38209 non-null object\n",
      "gender       38100 non-null object\n",
      "joinedAt     38152 non-null object\n",
      "location     32745 non-null object\n",
      "timezone     37773 non-null float64\n",
      "dtypes: float64(1), int64(1), object(5)\n",
      "memory usage: 2.0+ MB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中后 4 维特征(gender, joinedAt, location, timezone)都是有缺失值的, 还好缺失值都不算太多."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 统计总共有多少不同的 users (n_users)\n",
    "n_users = len(df['user_id'].unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "38209"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n_users"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "记录的数目等于用户的数目."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 特征工程"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# user_id 不作为聚类属性\n",
    "df = df.drop(['user_id'], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Medan  Indonesia                          4509\n",
       "Yogyakarta                                3092\n",
       "Phnom Penh                                2169\n",
       "Los Angeles  California                   1555\n",
       "                                          1475\n",
       "Santo Domingo  Dominican Republic         1442\n",
       "Toronto  Ontario                           696\n",
       "Phnom Penh  11                             631\n",
       "Tbilisi  Georgia                           540\n",
       "Phnom Pen  Phnum Penh  Cambodia            471\n",
       "San Francisco  California                  434\n",
       "Jogjakarta  Indonesia                      418\n",
       "Djokja  Yogyakarta  Indonesia              398\n",
       "Jakarta  Indonesia                         394\n",
       "Jakarta  04                                293\n",
       "Los Angeles  CA                            220\n",
       "Bekasi                                     211\n",
       "Medan  26                                  211\n",
       "Torrance  CA                               193\n",
       "undefined  undefined                       191\n",
       "Bandung  Indonesia                         179\n",
       "Miskolc  Hungary                           173\n",
       "Porto Alegre                               159\n",
       "Santo Domingo  05                          155\n",
       "Purwokerto  Jawa Tengah  Indonesia         154\n",
       "Surabaya  Indonesia                        154\n",
       "Ottawa  Ontario                            149\n",
       "New York  New York                         140\n",
       "Jombang  Jawa Timur  Indonesia             131\n",
       "Phoenix  Arizona                           128\n",
       "                                          ... \n",
       "Karlsruhe  Germany                           1\n",
       "Musquash  New Brunswick                      1\n",
       "Hilversum  07                                1\n",
       "Arctic Bay  Nunavut                          1\n",
       "Los Guayacanes  Narino  Colombia             1\n",
       "South Orange  New Jersey                     1\n",
       "Nglegok  Jawa Timur  Indonesia               1\n",
       "Kompong Thom                                 1\n",
       "Segno  Liguria  Italy                        1\n",
       "Phum Thmar Kol  Batdambang  Cambodia         1\n",
       "Maple  Ontario                               1\n",
       "Napa  California                             1\n",
       "Gazipur  Dhaka  Bangladesh                   1\n",
       "North Vancouver  British Columbia            1\n",
       "Kernersville  NC                             1\n",
       "Villa Altagracia                             1\n",
       "Bucaramanga  Santander                       1\n",
       "Fort Lauderdale  Florida                     1\n",
       "Cookstown                                    1\n",
       "Miami  Texas                                 1\n",
       "Edinburgh  United Kingdom                    1\n",
       "Valencia  07                                 1\n",
       "Nsukka  47                                   1\n",
       "South Lake Tahoe  California                 1\n",
       "Wawondula  Sulawesi Selatan  Indonesia       1\n",
       "Encantado  Rio Grande Do Sul  Brazil         1\n",
       "Pars  26                                     1\n",
       "Pasay City  Philippines                      1\n",
       "Boston  New York                             1\n",
       "Monterey  California                         1\n",
       "Name: location, Length: 2804, dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 查看 location 有多少取值及每个取值人数\n",
    "df['location'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "location 属离散特征, 考虑可能转为经纬度信息比较好, 但处理起来会很复杂. 此处直接简单抛弃."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>locale</th>\n",
       "      <th>birthyear</th>\n",
       "      <th>gender</th>\n",
       "      <th>joinedAt</th>\n",
       "      <th>timezone</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>id_ID</td>\n",
       "      <td>1993</td>\n",
       "      <td>male</td>\n",
       "      <td>2012-10-02T06:40:55.524Z</td>\n",
       "      <td>480.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>id_ID</td>\n",
       "      <td>1992</td>\n",
       "      <td>male</td>\n",
       "      <td>2012-09-29T18:03:12.111Z</td>\n",
       "      <td>420.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>en_US</td>\n",
       "      <td>1975</td>\n",
       "      <td>male</td>\n",
       "      <td>2012-10-06T03:14:07.149Z</td>\n",
       "      <td>-240.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>en_US</td>\n",
       "      <td>1991</td>\n",
       "      <td>female</td>\n",
       "      <td>2012-11-04T08:59:43.783Z</td>\n",
       "      <td>210.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>id_ID</td>\n",
       "      <td>1995</td>\n",
       "      <td>female</td>\n",
       "      <td>2012-09-10T16:06:53.132Z</td>\n",
       "      <td>420.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  locale birthyear  gender                  joinedAt  timezone\n",
       "0  id_ID      1993    male  2012-10-02T06:40:55.524Z     480.0\n",
       "1  id_ID      1992    male  2012-09-29T18:03:12.111Z     420.0\n",
       "2  en_US      1975    male  2012-10-06T03:14:07.149Z    -240.0\n",
       "3  en_US      1991  female  2012-11-04T08:59:43.783Z     210.0\n",
       "4  id_ID      1995  female  2012-09-10T16:06:53.132Z     420.0"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df.drop(['location'], axis=1)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 特征编码\n",
    "# 导入处理日期和时间的库\n",
    "import datetime\n",
    "# 导入 hash 算法库\n",
    "import hashlib\n",
    "# 导入处理多语言模块\n",
    "import locale\n",
    "# 导入集合模块\n",
    "from collections import defaultdict\n",
    "# 导入预处理中标准化模块\n",
    "from sklearn.preprocessing import normalize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 定义特征工程类\n",
    "class FeatureEng:\n",
    "    def __init__(self):\n",
    "        # 载入 locale\n",
    "        self.local_id_map = defaultdict(int)\n",
    "        for i, l in enumerate(locale.locale_alias.keys()):\n",
    "            self.local_id_map[l] = i+1\n",
    "            \n",
    "        # 载入 gender id 字典, 缺失值作为第 0 类\n",
    "        self.gender_id_map = defaultdict(int, {'NaN': 0, 'male': 1, 'female': 2})\n",
    "        \n",
    "    def get_locale_id(self, loc_str):\n",
    "        return self.local_id_map[loc_str.lower()]\n",
    "    \n",
    "    def get_gender_id(self, gender_str):\n",
    "        return self.gender_id_map[gender_str]\n",
    "    \n",
    "    def get_joined_year_month(self, date_string):\n",
    "        try:\n",
    "            date_time = datetime.datetime.strptime(date_string, '%Y-%m-%dT%H:%M:%S.%fZ')\n",
    "            return (date_time.year-2010)*12+date_time.month # 返回已注册月数\n",
    "        except:\n",
    "            return 0 # 缺失值填补 0\n",
    "        \n",
    "    def get_birth_year_int(self, birth_year):\n",
    "        try:\n",
    "            return 0 if birth_year=='None' else int(birth_year) # 缺失值填补 0\n",
    "        except:\n",
    "            return 0\n",
    "    \n",
    "    def get_timezone_int(self, timezone):\n",
    "        try:\n",
    "            return int(timezone)\n",
    "        except:\n",
    "            return 0 # 缺失值填补 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 特征工程类实例化\n",
    "FE = FeatureEng()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cols = ['LocaleId', 'BirthYearInt', 'GenderId', 'JoinedYearMonth', 'TimezoneInt']\n",
    "n_cols = len(cols)\n",
    "user_matrix = np.zeros((df.shape[0], n_cols), dtype=np.int) # 用户矩阵初始化为 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "user_matrix[:, 0] = df['locale'].apply(FE.get_locale_id) # 处理语言"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "user_matrix[:, 1] = df['birthyear'].apply(FE.get_birth_year_int) # 处理出生年"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "user_matrix[:, 2] = df['gender'].apply(FE.get_gender_id) # 处理性别"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "user_matrix[:, 3] = df['joinedAt'].apply(FE.get_joined_year_month) # 处理注册月数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "user_matrix[:, 4] = df['timezone'].apply(FE.get_timezone_int) # 处理时区"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>LocaleId</th>\n",
       "      <th>BirthYearInt</th>\n",
       "      <th>GenderId</th>\n",
       "      <th>JoinedYearMonth</th>\n",
       "      <th>TimezoneInt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.000036</td>\n",
       "      <td>0.000027</td>\n",
       "      <td>0.000019</td>\n",
       "      <td>0.000026</td>\n",
       "      <td>0.000036</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.000036</td>\n",
       "      <td>0.000027</td>\n",
       "      <td>0.000019</td>\n",
       "      <td>0.000026</td>\n",
       "      <td>0.000031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.000020</td>\n",
       "      <td>0.000027</td>\n",
       "      <td>0.000019</td>\n",
       "      <td>0.000026</td>\n",
       "      <td>-0.000018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.000020</td>\n",
       "      <td>0.000027</td>\n",
       "      <td>0.000038</td>\n",
       "      <td>0.000027</td>\n",
       "      <td>0.000016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.000036</td>\n",
       "      <td>0.000027</td>\n",
       "      <td>0.000038</td>\n",
       "      <td>0.000026</td>\n",
       "      <td>0.000031</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   LocaleId  BirthYearInt  GenderId  JoinedYearMonth  TimezoneInt\n",
       "0  0.000036      0.000027  0.000019         0.000026     0.000036\n",
       "1  0.000036      0.000027  0.000019         0.000026     0.000031\n",
       "2  0.000020      0.000027  0.000019         0.000026    -0.000018\n",
       "3  0.000020      0.000027  0.000038         0.000027     0.000016\n",
       "4  0.000036      0.000027  0.000038         0.000026     0.000031"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 归一化用户矩阵\n",
    "user_matrix = normalize(user_matrix, norm='l1', axis=0, copy=False)\n",
    "df_FE = pd.DataFrame(data=user_matrix, columns=cols)\n",
    "df_FE.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 聚类, 依 CH_score 选择聚类数目 K"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 评价聚类数目为 K 的模型的性能\n",
    "def K_clusters_analysis(K, dtrain):\n",
    "    print('K-Means begins with clusters: {}'.format(K))\n",
    "    # K-Means 训练\n",
    "    mb_kmeans = MiniBatchKMeans(n_clusters=K, batch_size=1000)\n",
    "    mb_kmeans.fit(dtrain)\n",
    "    \n",
    "    # K 值的评估标准: 轮廓系数 Silhouette Coefficient\n",
    "    CH_score = metrics.silhouette_score(dtrain, mb_kmeans.predict(dtrain))\n",
    "    print('CH_score: {}'.format(CH_score))\n",
    "    \n",
    "    return CH_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "K-Means begins with clusters: 20\n",
      "CH_score: 0.6944568771586038\n",
      "K-Means begins with clusters: 40\n",
      "CH_score: 0.733002231664124\n",
      "K-Means begins with clusters: 80\n",
      "CH_score: 0.6177341288318459\n"
     ]
    }
   ],
   "source": [
    "# 设置超参数(聚类数目 K)搜索范围\n",
    "Ks = [20, 40, 80]\n",
    "CH_scores = []\n",
    "for K in Ks:\n",
    "    ch = K_clusters_analysis(K, df_FE)\n",
    "    CH_scores.append(ch)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10bd58c88>]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xm81nP+//HHq+UUxdjKNCVlZEnN\nxJzCRKsU3xTGUqYZ21fjZxmMGNlLxljKMhokEoMmRE1Fiih7J8loI2V0hEIZa+vr98fr6us4yrk6\n2+danvfbrVvn+pzPdc7rXdd5Xu/z/rw/77e5OyIikh9qJF2AiIhUH4W+iEgeUeiLiOQRhb6ISB5R\n6IuI5BGFvohIHlHoi4jkEYW+iEgeUeiLiOSRWkkXUNouu+zizZo1S7oMEZGsMnv27E/cvUFZ52Vc\n6Ddr1oyioqKkyxARySpm9p90ztPwjohIHlHoi4jkEYW+iEgeUeiLiOQRhb6ISB5R6IuI5BGFvohI\nHlHoS5V64QV4/PGkqxCRTRT6UiU+/xz+8Ac49FA49li44YakKxIRyMA7ciX7TZoUgf/hhzBgABQX\nw5//DGvWwBVXJF2dSH5T6Eul+eQTOP98ePBBaNUqhnXatoUNG6CgAK68EtauhcGDwSzpakXyk0Jf\nKswdxo6Fc8+F1avh6qth4MAIeoCaNWHUqHg8ZEj0+K+/XsEvkgSFvlTI8uVw1lkwfnz06u+9N3r5\npdWoAXfdFcF/443R47/5ZgW/SHVT6Eu5uEfAX3hh9NxvuimGdmrW3PJzatSA22+P4L/llgj+22+P\n4yJSPRT6stWWLoX+/WHaNOjYEUaOhD33TO+5ZjBsGNSpE0M8a9fGbwA/9mYhIpVHoS9p27gxeuYD\nB0ZI33knnHHG1vfUzeC66yL4Bw+O4L/3XqilV6NIldOPmaRl4UI4/XR46SU48sgI/N12K//XM4NB\ng6B27ZjGuXYtPPBAPBaRqqPQlx+1bl2M1w8aBPXqRTD/9reVdwH28sujx3/xxfG9Hn74u1k/IlL5\nFPqyRXPmRO9+zhw4/nj4299g110r//tcdFEE/fnnw3HHwSOPxBuBiFQ+zZuQH/j2W7jsspiC+eGH\nMG5czMOvisDf5Lzz4I474F//gt694Ztvqu57ieQz9fTle156KXr3CxfCqafC0KGw447V873PPDN6\n/P/7v9CzJ0yYEENKIlJ51NMXAL78MnrbhxwSvewpU2JGTXUF/iannQb33w/PPQdHHAFffFG9318k\n1yn0hWnToHXrGLM/5xx46y04/PDk6unXDx56KH7r6N49VuwUkcqh0M9jq1fHUE63bnHhdMYMuO02\nqF8/6crgxBPjOkJRUdS3alXSFYnkBoV+nho/Hlq2hNGj4ZJL4I03Ymgnkxx7bFxEnjsXunSJVTxF\npGIU+nlmxQro0weOPhoaNoTXXou7Y+vWTbqyzdt0QXfhQujcGT7+OOmKRLKbQj9PuMc69y1bxjr3\nQ4bArFlwwAFJV1a27t1jY5YlS6BTp5hGKiLlk1bom1kPM1tkZovN7JLNfP5mM3sj9edtM1udOt7G\nzF42s3lm9qaZnVjZDZCyFRfDUUfFBdIWLeJmq8suy64lD7p0gSefjLZ07Bh/i8jWKzP0zawmMBw4\nAmgJ9DWzliXPcfcL3L2Nu7cB/gaMS33qa+D37r4f0AO4xcx2qMwGyJZt3AgjRsB++8H06bF+/Qsv\nRG8/G3XoAE8/HUM8HTrAe+8lXZFI9kmnp98OWOzuS9x9LTAG6P0j5/cFHgZw97fd/Z3Ux8uBFUCD\nipUs6Xj3XejaNfaqLSyEf/+77PXus8HBB8cU01Wrosf/7rtJVySSXdIJ/cbAshKPi1PHfsDMdgea\nA89u5nPtgAJAP6ZVaMOGWK++dWt4/XW4++4IyT32SLqyytO2LTz7LHz1VfT4Fy1KuiKR7JFO6G9u\nPUXfwrl9gEfdfcP3voBZI+AB4FR33/iDb2DW38yKzKxo5cqVaZQkmzNvHvz617Gb1WGHwfz5saRB\nLm5JuP/+MWS1fn30+OfPT7oikeyQTugXAyVXTm8CLN/CuX1IDe1sYmbbA5OAy939lc09yd1HuHuh\nuxc2aKDRn621dm1sRrL//jHD5eGHYx5+483+PpY7WreO5Rpq1IhZPW++mXRFIpkvndCfBbQws+Zm\nVkAE+4TSJ5nZ3sCOwMsljhUAjwP3u/sjlVOylFRUFGP2V10VyxLPnx/z8HOxd785++4Lzz8fdxR3\n7hxDWiKyZWWGvruvB84BpgALgLHuPs/MBptZrxKn9gXGuHvJoZ8TgA7AKSWmdLapxPrz1jffwJ//\nDAceCJ9+GjcwPfQQ5OMvSi1axBIS220XUztffTXpikQyl30/o5NXWFjoRUVFSZeR0WbOjDVz3nkn\n9qi94QbYQRNhef/96O2vXAmTJ2feshIiVcnMZrt7YVnn6Y7cLPLFF3D22TFjZf16eOaZmIevwA9N\nm0aPv1Ej6NEjxvtF5PsU+lliyhRo1Sp2lzr//Jh336VL0lVlnsaNY4x/991jA/dp05KuSCSzKPQz\n3GefwSmnRM+1Xj148cW4s1Y7Sm3ZT38avfwWLWLBtsmTk65IJHMo9DPYY4/FkgkPPgiXXx5r5hx8\ncNJVZYcGDeIGrv32ixVFx49PuiKRzKDQz0AffRTTL487LoYrZs2Ca66JaYmSvp13juse++8f/5aP\nPpp0RSLJU+hnEPfY1KRlS5g4Mda5f/VVaKNJruW2ww4wdWpMbe3TJ6a1iuQzhX6GeP/9uPB4yikR\n+nPnxo5WtWolXVn22357eOopOPTQWF569OikKxJJjkI/YRs3wt//HmPPM2fG5uQzZsDeeyddWW6p\nXz82YjnsMDj11FiITiQfqR+ZoLffjgXRZs6Mzb9HjIBmzZKuKndtu23cufyb30D//rFm0dlnJ12V\nSPVSTz8B69fHXbS//GXMtx81KubhK/CrXt26sdl6795wzjkx/VUkn6inX83efBNOOw1mz4ZjjoHh\nw+MOUqk+derAI4/ASSfBn/4Ea9bE9RORfKDQryZr1sC118aMnJ12grFjYxphvqyGmWlq144lqAsK\nYODAGOq54gr9f0juU+hXg1dfjd79/Pnwu9/FkMLOOyddldSqBfffH8F/1VXxxjxkiIJfcptCvwp9\n/XXcSXvLLXGT1aRJMS1TMkfNmnDPPdHz/8tfIvhvvFHBL7lLoV9Fpk+PmTlLlsCZZ8L118d8cck8\nNWrAnXdGj3/o0BjqufVWBb/kJoV+Jfv8c7j44ph++fOfR/h36pR0VVKWGjXiHok6dWJj+bVr4/6J\nGprfJjlGoV+JJk6MXv2HH8KAATBoUMwNl+xgBjfdFMF/3XUR/HffHUNAIrlCoV8JPvkEzjsv1nVp\n1Srmgbdrl3RVUh5mMcuqoCDetNeuhfvu03IYkjv0Uq4A95h6ee65sHp1zAC59NIIDMleZnD11fH/\neNllsG4d/OMfcbFXJNsp9Mtp+XI466xYp71t25gB0rp10lVJZbr00hjqGTAggn/MGL2hS/bTZaqt\n5B4B37JlLJ1w443w0ksK/Fx14YVw223w+ONw7LHw7bdJVyRSMerpb4WlS2OhrmnTYnPykSNjSz7J\nbeeeGz3+P/wh1ux5/HFdoJfspZ5+GjZsiN5eq1Zxd+0dd8RUTAV+/ujfH+69NzZk6dkTvvoq6YpE\nykc9/TIsWBA3Wb30EhxxBNx1F+y2W9JVSRJOPTUu5p58cmxUP3kybLdd0lWJbB319Ldg3bq4Lb9N\nG1i4MNZomTRJgZ/v+vWLhdpefhkOPzxmbYlkk7RC38x6mNkiM1tsZj9YhNbMbjazN1J/3jaz1SU+\nd7KZvZP6c3JlFl9V5syJefaXXQa9en23UJpuyxeAE06ITdZnz46duD77LOmKRNJXZuibWU1gOHAE\n0BLoa2YtS57j7he4ext3bwP8DRiXeu5OwFXAgUA74Coz27Fym1B5vv02pum1bQsffQSPPRbrru+6\na9KVSaY5+ui4oPvWW9ClC6xcmXRFIulJp6ffDljs7kvcfS0wBuj9I+f3BR5OfdwdmOrun7n7KmAq\n0KMiBVeVl16C/feP2+9/97vo3R97bNJVSSb7n/+J7RcXLYLOnaOjIJLp0gn9xsCyEo+LU8d+wMx2\nB5oDz27tc5Py5ZexhMIhh8RSyE89FdsX7pixv49IJjn88LjWs3RpLKy3fHnSFYn8uHRCf3Mj2b6F\nc/sAj7r7hq15rpn1N7MiMytaWY2/J0+dGjdV3XZbbJD91lvQvXu1fXvJEV26RGfhgw+gY0dYtqzs\n54gkJZ3QLwZKzllpAmypP9OH74Z20n6uu49w90J3L2zQoEEaJVXM6tVw+unRSysogJkzY1ldTb+T\n8jr00OhErFwZN+4tXZp0RSKbl07ozwJamFlzMysggn1C6ZPMbG9gR+DlEoenAIeb2Y6pC7iHp44l\n5oknYgmF0aNjM+w33oihHZGKOuiguFv788+jx794cdIVifxQmaHv7uuBc4iwXgCMdfd5ZjbYzHqV\nOLUvMMbdvcRzPwOuId44ZgGDU8eq3YoVcOKJcMwx0LBh3Fl73XWwzTZJVCO5qrAQnn0WvvkmevwL\nFyZdkcj3WYmMzgiFhYVeVFRUaV/PPda5P+88+OILuOIK+POftUyuVK233oo5/BC9/1atkq1Hcp+Z\nzXb3wrLOy+k7cpctg6OOirso99wzbrq6/HIFvlS9Vq3guediu8XOnWHu3KQrEgk5GfobN8YaOfvt\nF79qDxsGL74YY/ki1WWffWDGjBhC7NwZKvEXWJFyy7nQf/dd6No19qpt2zZ+zb7gAu1zKsnYc094\n/nn4yU/idfnKK0lXJPkuZ0J/w4bo0bduDa+/DiNGxFjqHnskXZnku+bNI/gbNIBu3eCFF5KuSPJZ\nzoT+0qUwcGD0pubNgzPO0AJpkjmaNo2hnsaN4wbA6dOTrkjyVc6E/p57xpz7CROgSZOkqxH5oZ/9\nLHr8zZvDkUfC008nXZHko5wJfYB991XvXjLbrrtGL3/vvWNm2aRJSVck+SanQl8kGzRoELPKWreO\nmwUffzzpiiSfKPRFErDTTjHR4Fe/guOPh7Fjk65I8oVCXyQhO+wQ4/oHHwx9+8KDDyZdkeQDhb5I\ngrbbLpZl7tgxNu8ZNSrpiiTXKfRFElavHkycGHP4Tzst7iYXqSoKfZEMsO22MH58bMF45pmxv4NI\nVVDoi2SIunVh3LjYdP2Pf4ShQ5OuSHKRQl8kgxQUxEyeE06AAQPgL39JuiLJNbWSLkBEvq927ZjJ\nU1AAl10Ga9fCVVfpxkOpHAp9kQxUqxbcd1+8AQwaBGvWRK9fwS8VpdAXyVA1a8LIkdHj/+tfo8d/\n000KfqkYhb5IBqtRA+64I4J/2LAI/ltvjeMi5aHQF8lwZhH0depET3/NGrjzTgW/lI9CXyQLmMEN\nN0TwX3tt9PjvuUc7wsnWU+iLZAkzGDIkhnquugrWrYPRo+Oir0i69HIRyTJXXhnBP3Bg9Pgfeihm\n+YikQ6EvkoUuuSSGev70p+jx//Of8VikLLoUJJKlLrgAbr891uw59lj49tukK5JskFbom1kPM1tk\nZovN7JItnHOCmc03s3lm9lCJ4zekji0ws9vMNMtYpLKcfTaMGAFPPhnbL379ddIVSaYrc3jHzGoC\nw4FuQDEwy8wmuPv8Eue0AAYC7d19lZk1TB3/NdAe+EXq1BeAjsBzldkIkXx2xhkxpn/aabFK57/+\nBfXrJ12VZKp0evrtgMXuvsTd1wJjgN6lzjkDGO7uqwDcfUXquAN1gQKgDlAb+LgyCheR75xyCvzj\nHzBzJvToAf/9b9IVSaZKJ/QbA8tKPC5OHStpL2AvM3vRzF4xsx4A7v4yMB34MPVnirsvqHjZIlLa\nSSfBmDHw6qtw+OGwenXSFUkmSif0NzcG76Ue1wJaAJ2AvsBIM9vBzPYE9gWaEG8UXcysww++gVl/\nMysys6KVK1duTf0iUsJxx8Gjj8Lrr0PXrvDpp0lXJJkmndAvBnYr8bgJsHwz54x393XuvhRYRLwJ\nHAO84u5fuvuXwJPAQaW/gbuPcPdCdy9s0KBBedohIim9e8MTT8C8edClC6xYUfZzJH+kE/qzgBZm\n1tzMCoA+wIRS5zwBdAYws12I4Z4lwPtARzOrZWa1iYu4Gt4RqWJHHhkXdN95Bzp3ho8+SroiyRRl\nhr67rwfOAaYQgT3W3eeZ2WAz65U6bQrwqZnNJ8bwL3L3T4FHgXeBfwNzgbnu/q8qaIeIlNKtG0ye\nDP/5D3TsCB98kHRFkgnMvfTwfLIKCwu9qKgo6TJEcsaLL8IRR0DDhvDss9C0adIVSVUws9nuXljW\nebojVyTHtW8PU6fCJ59Ahw6wZEnSFUmSFPoieeDAA6OX/8UXMdTzzjtJVyRJUeiL5IkDDojg//bb\nCP4FmlKRlxT6Innkl7+E556DjRuhUyd4662kK5LqptAXyTP77QfPPx+br3TqBHPmJF2RVCeFvkge\n2ntvmDED6tWLG7hmzUq6IqkuCn2RPPXzn0ePf8cd4bDD4OWXk65IqoNCXySPNWsWwb/rrrFI24wZ\nSVckVU2hL5Lndtstgr9Jk7iJ65lnkq5IqpJCX0Ro1Chm9eyxB/TsCVOmJF2RVBWFvogAMcQzfTrs\nsw/06hULtknuUeiLyP/ZZZcY3vnFL2Kz9XHjkq5IKptCX0S+Z6edYNo0aNsWTjgB/vnPpCuSyqTQ\nF5Ef+MlPYly/ffvYhvGBB5KuSCqLQl9ENmu77WI9/k6d4OST4d57k65IKoNCX0S2qF49mDgx5vCf\nfjrccUfSFUlFKfRF5Edts03suduzJ5x1Ftx6a9IVSUUo9EWkTHXrwmOPxYye88+HG29MuiIpL4W+\niKSloADGjIE+feDii2HIkKQrkvKolXQBIpI9ateOmTy1a8MVV8DatTBoEJglXZmkS6EvIlulVi0Y\nNSqC/5prIvivu07Bny0U+iKy1WrWhLvvhjp14PrrYc0aGDZMwZ8NFPoiUi41asDw4THWf8st0eP/\n29/iuGQuhb6IlJsZ3Hxz9PhvuCGC/667FPyZTKEvIhViBn/9a/T4hwyJ4L/33hgCksyj0BeRCjOL\ni7oFBXDllRH8DzwQF30ls6T1S5iZ9TCzRWa22Mwu2cI5J5jZfDObZ2YPlTje1MyeNrMFqc83q5zS\nRSTTXHFFXNjdNJ9/7dqkK5LSynwfNrOawHCgG1AMzDKzCe4+v8Q5LYCBQHt3X2VmDUt8ifuBa919\nqpnVBzZWagtEJKNcfHH0+C+4AI47Dh55JMb8JTOk09NvByx29yXuvhYYA/Qudc4ZwHB3XwXg7isA\nzKwlUMvdp6aOf+nuX1da9SKSkc4/H/7+99h96+ij4Ztvkq5INkkn9BsDy0o8Lk4dK2kvYC8ze9HM\nXjGzHiWOrzazcWY2x8xuTP3m8D1m1t/MisysaOXKleVph4hkmP/3/2DkyFiX/6ij4Kuvkq5IIL3Q\n39ztFl7qcS2gBdAJ6AuMNLMdUscPBQYAbYE9gFN+8MXcR7h7obsXNmjQIO3iRSSznX463Hdf7L17\n5JHwxRdJVyTphH4xsFuJx02A5Zs5Z7y7r3P3pcAi4k2gGJiTGhpaDzwBHFDxskUkW/z+9/Dgg/Di\ni9CjB3z+edIV5bd0Qn8W0MLMmptZAdAHmFDqnCeAzgBmtgsxrLMk9dwdzWxT970LMB8RySt9+sRe\nu6+9Bt26wapVSVeUv8oM/VQP/RxgCrAAGOvu88xssJn1Sp02BfjUzOYD04GL3P1Td99ADO08Y2b/\nJoaK7q6KhohIZvvNb2DcOJg7F7p2hU8+Sbqi/GTupYfnk1VYWOhFRUVJlyEiVeSpp2JGz157wbRp\n0LBh2c+RspnZbHcvLOs8rZAhItWqRw+YNAkWL45N1z/8MOmK8otCX0SqXdeu8OST8P770LEjFBcn\nXVH+UOiLSCI6doSnn4aPP46P//OfpCvKDwp9EUnMr38NU6fCZ59Bhw7w7rtJV5T7FPoikqh27eCZ\nZ+DLL6PH//bbSVeU2xT6IpK4Aw6A556LVTk7doT5upunyij0RSQjtG4dwQ8xq+fNN5OsJncp9EUk\nY7RsCc8/H0szd+4Mr7+edEW5R6EvIhllr71gxgyoXz+mdr72WtIV5RaFvohknD32iODfaSc47LBY\nrE0qh0JfRDLS7rvHUE+jRtC9e3wsFafQF5GM1aRJXNzdfXc44ohYq0cqRqEvIhmtUaPYhGXPPaFn\nz1i+QcpPoS8iGa9hwwj+li1jhc4JpXf0kLQp9EUkK+y8c9y526ZNrM3/2GNJV5SdFPoikjV23DHW\n6mnXDk48ER5+OOmKso9CX0Syyvbbw5QpcMgh0K8fjB6ddEXZRaEvIlmnfn2YPBm6dIFTT4WRI5Ou\nKHso9EUkK227bVzQ7d4dzjgD/v73pCvKDgp9Ecla22wDTzwBvXrB2WfDLbckXVHmU+iLSFarUwce\neSRm9FxwAVx/fdIVZTaFvohkvYICGDMG+vaFSy6Ba65JuqLMVSvpAkREKkOtWvDAA1C7Nlx5JaxZ\nE+FvlnRlmUWhLyI5o2ZNGDUqev7XXhs7cV1/vYK/JIW+iOSUGjXgrrsi+G+8MYL/5psV/JukNaZv\nZj3MbJGZLTazS7ZwzglmNt/M5pnZQ6U+t72ZfWBmt1dG0SIiP6ZGDbj99riwe+utcNZZsHFj0lVl\nhjJ7+mZWExgOdAOKgVlmNsHd55c4pwUwEGjv7qvMrGGpL3MNoNWwRaTamMHQodHjv/766PGPGBFD\nQPksneGddsBid18CYGZjgN5Ayf3qzwCGu/sqAHdfsekTZvYrYFfgKaCwkuoWESmTGVx3XUzrHDw4\ngn/UqLjom6/SaXpjYFmJx8XAgaXO2QvAzF4EagJXu/tTZlYDGAr8Duha8XJFRLaOGQwaFD3+yy+H\ndeu+m+WTj9IJ/c1d/vDNfJ0WQCegCTDTzFoB/YDJ7r7MfuQqipn1B/oDNG3aNI2SRES2zmWXRY//\noouixz9mTLwR5Jt0Qr8Y2K3E4ybA8s2c84q7rwOWmtki4k3gYOBQMzsLqA8UmNmX7v69i8HuPgIY\nAVBYWFj6DUVEpFIMGBBBf955cQfvI49A3bpJV1W90pm9MwtoYWbNzawA6AOU3rfmCaAzgJntQgz3\nLHH337p7U3dvBgwA7i8d+CIi1emPf4Q77oCJE6F3b/jmm6Qrql5lhr67rwfOAaYAC4Cx7j7PzAab\nWa/UaVOAT81sPjAduMjdP62qokVEKuLMM+Gee2JDlp494auvkq6o+ph7Zo2mFBYWelFRUdJliEge\n+Mc/4OSToX17mDQJttsu6YrKz8xmu3uZMyS14JqI5K1+/eChh+Cll2Jd/s8/T7qiqqfQF5G8duKJ\ncUG3qAgOOww++yzpiqqWQl9E8t4xx8C4cfDmm9C1K3zySdIVVR2FvogIcUF3wgRYuBA6d4aPP066\noqqh0BcRSenePS7oLlkCnTrB8tJ3JOUAhb6ISAldusBTT0FxMXTsCMuWlf2cbKLQFxEp5dBD4emn\nYcWKCP733ku6osqj0BcR2YyDD4Zp02DVKujQARYvTrqiyqHQFxHZgrZtYfp0+Prr6PEvWpR0RRWn\n0BcR+RFt2sBzz8H69RH88+YlXVHFKPRFRMrQqhU8/3xsw9ipE8ydm3RF5afQFxFJwz77RPDXrRvz\n+GfPTrqi8lHoi4ikqUULmDEDtt8+7tx99dWkK9p6Cn0Rka3QvHkE/y67QLdu8MILSVe0dRT6IiJb\nqWnTGOr52c+gR4+40JstFPoiIuXQuHGE/e67w5FHxoYs2UChLyJSTj/9aQR/ixZw1FEweXLSFZVN\noS8iUgENGsCzz8J++8HRR8P48UlX9OMU+iIiFbTzzvDMM3DAAXDccbEpS6ZS6IuIVIIddohF2g46\nCPr0iW0YM5FCX0Skkmy/PTz5ZCzQ1q8f3Hdf0hX9kEJfRKQS1a8fG7EcdhiceiqMGJF0Rd+n0BcR\nqWTbbhtbLx55JPzhD3D77UlX9B2FvohIFahbNzZb790bzj0Xhg1LuqKg0BcRqSJ16sRMnuOPhwsv\nhOuuS7qiNEPfzHqY2SIzW2xml2zhnBPMbL6ZzTOzh1LH2pjZy6ljb5rZiZVZvIhIpqtdO2bynHQS\nXHopDBoE7snVU6usE8ysJjAc6AYUA7PMbIK7zy9xTgtgINDe3VeZWcPUp74Gfu/u75jZz4DZZjbF\n3VdXektERDJUrVpw//1QUABXXw1r18KQIWCWQC1pnNMOWOzuSwDMbAzQG5hf4pwzgOHuvgrA3Vek\n/n570wnuvtzMVgANAIW+iOSVmjXhnnsi+P/yF1izBm68sfqDP53QbwwsK/G4GDiw1Dl7AZjZi0BN\n4Gp3f6rkCWbWDigA3i13tSIiWaxGDbjzzgj+oUOjx3/rrdUb/OmE/ubKKT0iVQtoAXQCmgAzzazV\npmEcM2sEPACc7O4bf/ANzPoD/QGaNm2advEiItnGDG67LS7yDh0aPf477og3hOqQTugXA7uVeNwE\nWL6Zc15x93XAUjNbRLwJzDKz7YFJwOXu/srmvoG7jwBGABQWFiZ4iUNEpOqZxdBOQUHM6Fm7FkaO\njCGgqpZO6M8CWphZc+ADoA9wUqlzngD6AveZ2S7EcM8SMysAHgfud/cMXoJIRKR6mcG110aPf9PF\n3dGj46JvVSrzy7v7ejM7B5hCjNff6+7zzGwwUOTuE1KfO9zM5gMbgIvc/VMz6wd0AHY2s1NSX/IU\nd3+jKhojIpJNzOCqq6LHf+mlsG4dPPxw1fb4zZOcMLoZhYWFXlRUlHQZIiLVatgwWLUKrrmmfM83\ns9nuXljWeVX8i4SIiKTjT3+qnu+jZRhERPKIQl9EJI8o9EVE8ohCX0Qkjyj0RUTyiEJfRCSPKPRF\nRPKIQl9EJI9k3B25ZrYS+E8FvsQuwCeVVE6ScqUdoLZkqlxpS660AyrWlt3dvUFZJ2Vc6FeUmRWl\ncytypsuVdoDakqlypS250g6onrZoeEdEJI8o9EVE8kguhv6IpAuoJLnSDlBbMlWutCVX2gHV0Jac\nG9MXEZEty8WevoiIbEHWhr44SVmCAAAD6ElEQVSZ7WZm081sgZnNM7PzUsd3MrOpZvZO6u8dk661\nLGZW18xeM7O5qbYMSh1vbmavptryz9T2kxnPzGqa2Rwzm5h6nK3teM/M/m1mb5hZUepY1r2+AMxs\nBzN71MwWpn5mDs7GtpjZ3qn/j01//mtm52dpWy5I/by/ZWYPp3Kgyn9Wsjb0gfXAhe6+L3AQcLaZ\ntQQuAZ5x9xbAM6nHmW4N0MXdfwm0AXqY2UHA9cDNqbasAk5PsMatcR6woMTjbG0HQGd3b1NiGl02\nvr4AbgWecvd9gF8S/z9Z1xZ3X5T6/2gD/Ar4mtiHO6vaYmaNgT8Che7eitiKtg/V8bPi7jnxBxgP\ndAMWAY1SxxoBi5KubSvbsS3wOnAgcZNGrdTxg4EpSdeXRv1NiB+6LsBEwLKxHala3wN2KXUs615f\nwPbAUlLX8LK5LaXqPxx4MRvbAjQGlgE7ETsYTgS6V8fPSjb39P+PmTUD9gdeBXZ19w8BUn83TK6y\n9KWGRN4AVgBTgXeB1e6+PnVKMfFCyXS3ABcDG1OPdyY72wHgwNNmNtvM+qeOZePraw9gJTAqNew2\n0szqkZ1tKakP8HDq46xqi7t/ANwEvA98CHwOzKYaflayPvTNrD7wGHC+u/836XrKy903ePzK2gRo\nB+y7udOqt6qtY2Y9gRXuPrvk4c2cmtHtKKG9ux8AHEEMH3ZIuqByqgUcANzh7vsDX5Hhwx9lSY11\n9wIeSbqW8khdc+gNNAd+BtQjXmelVfrPSlaHvpnVJgL/QXcflzr8sZk1Sn2+EdFzzhruvhp4jrhO\nsYOZbdq8vgmwPKm60tQe6GVm7wFjiCGeW8i+dgDg7stTf68gxo3bkZ2vr2Kg2N1fTT1+lHgTyMa2\nbHIE8Lq7f5x6nG1tOQxY6u4r3X0dMA74NdXws5K1oW9mBtwDLHD3YSU+NQE4OfXxycRYf0YzswZm\ntkPq422IF8QCYDpwXOq0jG+Luw909ybu3oz41ftZd/8tWdYOADOrZ2bbbfqYGD9+iyx8fbn7R8Ay\nM9s7dagrMJ8sbEsJffluaAeyry3vAweZ2bapLNv0f1LlPytZe3OWmR0CzAT+zXfjx5cS4/pjgabE\nP+zx7v5ZIkWmycx+AYwmruDXAMa6+2Az24PoMe8EzAH6ufua5CpNn5l1Aga4e89sbEeq5sdTD2sB\nD7n7tWa2M1n2+gIwszbASKAAWAKcSuq1Rva1ZVviIuge7v556ljW/b+kpmafSMxEnAP8LzGGX6U/\nK1kb+iIisvWydnhHRES2nkJfRCSPKPRFRPKIQl9EJI8o9EVE8ohCX0Qkjyj0RUTyiEJfRCSP/H/4\nx4lxWoNr4gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x109900f60>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 绘制不同聚类数目下的模型得分\n",
    "plt.plot(Ks, CH_scores, 'b-')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "K = 40 时得分最高."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 评价聚类数目为 K 的模型的性能\n",
    "def K_clusters_analysis(K, dtrain):\n",
    "    print('K-Means begins with clusters: {}'.format(K))\n",
    "    # K-Means 训练\n",
    "    mb_kmeans = MiniBatchKMeans(n_clusters=K, batch_size=1000)\n",
    "    mb_kmeans.fit(dtrain)\n",
    "    \n",
    "    # K 值的评估标准: 轮廓系数 Calinski Harabaz Index\n",
    "    CH_score = metrics.calinski_harabaz_score(dtrain, mb_kmeans.predict(dtrain))\n",
    "    print('CH_score: {}'.format(CH_score))\n",
    "    \n",
    "    return CH_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "K-Means begins with clusters: 20\n",
      "CH_score: 44580.83243603768\n",
      "K-Means begins with clusters: 40\n",
      "CH_score: 45207.990672963206\n",
      "K-Means begins with clusters: 80\n",
      "CH_score: 35526.60589892283\n"
     ]
    }
   ],
   "source": [
    "# 设置超参数(聚类数目 K)搜索范围\n",
    "Ks = [20, 40, 80]\n",
    "CH_scores = []\n",
    "for K in Ks:\n",
    "    ch = K_clusters_analysis(K, df_FE)\n",
    "    CH_scores.append(ch)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10ba89be0>]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl4lOW9xvHvL5NM2Am7SITEggta\n1xSxtgTCvq8WUCBYl1OPvWz1tC6nVY9rXQuuVSsQ9rCIgohAgCS4osGlLogiBokiYFlkzUB4zh95\nsYEECJDkncncn+uay5lnnknux8zMnXeZYM45RERESorxO4CIiIQflYOIiJSichARkVJUDiIiUorK\nQURESlE5iIhIKSoHEREpReUgIiKlqBxERKSUWL8DnKjGjRu7pKQkv2OIiESMlStX/uCca1KeuRFb\nDklJSeTl5fkdQ0QkYpjZuvLO1W4lEREpReUgIiKlqBxERKQUlYOIiJSichARkVJUDiIiUorKQURE\nSonYzzlIZNsV2kX+tnzyt+Xz9bavqR9fnxHnjcDM/I4mIqgcpJLs2bfnpzf/kiVw8Prm3ZtLPWbF\ntyt4oucTxJg2aEX8pnKQE7J3/16+2f5N8Zv+Vu9Nf/t/rm/ctfGQ+cFAkKSEJJISkrjwlAtJbpD8\n0+2khCTGvD2Gh996mML9hTzX9zkVhIjPVA5SplBR6Kc3/7J+8/9ux3eHzI+LiaNl/ZYkN0im7xl9\nSUpIOqQATqlzylHf8B/s8iDxsfHcu/xe9h3Yx7h+4wjEBCp7mSJyBCqHKLWvaB8FPxaUetM/eP3b\nH7/F4X6aH7AALeu3JCkhie4/605ywn/e+JMbJNO8TvOTejM3M+7pdA/BQJA7su8gVBRi0sBJxMbo\nKSriB73yqqmiA0U/vfkfXgD52/JZ/+N6DrgDP82PsRgS6yWSnJBM5+TOxW/6JQqgRb0WVfJG/dcO\nfyUYCHLrklsJFYWYPng6cYG4Sv++InIolUOEKjpQxIadGw7d51+iBNb/uJ79B/b/NN8wWtRrQVJC\nEr9u9etDf/NPSCaxXmLYvAnfctktBANBblp0E0NmDWHmkJnEx8b7HUskqqgcwtQBd4Dvd37/nzf9\nEgd987fls27bOvYd2HfIY5rXaU5yg2QuPe1Shtcffsh+/9PqnRZRb7B/bP9H4gPx/PeC/2bgjIG8\n+JsXqRlX0+9YIlFD5eAT5xybdm064j7/ddvWUVhUeMhjmtVuRlJCEhc3v5ghZw85ZJ9/y/otqRFb\nw6fVVI7rf3E9wUCQa1+5ln6Z/Zg7bC614mr5HUskKqgcKolzjh92/3DEff752/LZs3/PIY9pXKsx\nSQlJnN/sfPqf2f+Q/f6tElpF5Rvj1RddTTAQZPTc0fSa2ov5V8ynTrCO37FEqj2VwwlyzrF179Yj\n7vPP35bPrn27DnlMw5oNSUpI4uwmZ9OrTa9D9vm3SmilN70jGHn+SOICcYyYM4LuU7rz2pWvUS++\nnt+xRKo1lcNRbNu7rcx9/gev7wjtOGR+vfh6JCck06ZRG7qe3vWQff6t6reifo36Pq0k8g07dxhx\nMXEMe3EYXSd3ZeGVC2lQs4HfsUSqraguhx2FO464zz9/Wz7b9m47ZH6dYJ2fdvN0Sup0yCd8kxsk\nk1AjwaeVRIfBbQczJzCHIbOG0HlSZ7JGZtGoViO/Y4lUS+acO/asMJSSkuLy8vKO6zEH3AGGzh7K\n2q1ryd+Wz5Y9Ww65v1ZcrVLn95e83rBmQ/1huDCwcM1CBs4YSJuGbVgyaglNazf1O5JIRDCzlc65\nlPLMjaothxiLYcOODTSt3ZR2p7Y75M87JCck07hWY735R4AerXswf/h8+k7vS8eMjiwdtZTmdZv7\nHUukWomqLQepXnLzc+k9rTct6rVg2ahltKjXwu9IImHteLYc9KcvJWKlJqWyeORivt/5PR0yOrBu\n2zq/I4lUGyoHiWi/PO2XLBm5hC17ttAhowNrt671O5JItaBykIj3ixa/YNmoZewK7aLDhA588e8v\n/I4kEvFUDlItXNj8QrLTswkVhUjNSGXV5lV+RxKJaCoHqTZ+3uzn5IzOASA1I5WPN37sbyCRCKZy\nkGqlbZO25I7OJRgI0mliJz7Y8IHfkUQiUrnLwcwCZvaBmc0/bPxJM9tZ4na8mc0wszVmtsLMkkrc\nd7s3vtrMupcY7+GNrTGz205uSRLtzmh0BsuvWk6dYB3SJqXx7rfv+h1JJOIcz5bDH4BDduSaWQpw\n+N+MuBrY6pxrDYwBHvLmtgWGAecAPYBnvMIJAE8DPYG2wHBvrsgJO73B6eSOzqVhzYZ0mdSFt9a/\n5XckkYhSrnIws0SgN/BCibEA8Ahwy2HT+wMTveuzgc5W/LHj/kCmc67QOfc1sAZo513WOOfWOudC\nQKY3V+SktEpoRe7oXJrXbU63yd1Yvm6535FEIkZ5txzGUlwCB0qM/R6Y55zbcNjcFsB6AOfcfmA7\n0KjkuKfAGzvSuMhJS6yXSE56Dq0SWtFjSg+Wrl3qdySRiHDMcjCzPsAm59zKEmOnApcDT5b1kDLG\n3AmMl5XlOjPLM7O8zZs3Hyu6CADN6zYnOz2b1g1b02d6HxauWeh3JJGwV54th8uAfmaWT/EunzTg\nU6A1sMYbr2Vma7z5BcBpAGYWC9QHtpQc9yQC3x1lvBTn3PPOuRTnXEqTJk3Ksz4RAJrWbkp2ejZt\nm7Slf2Z/Xln9it+RRMLaMcvBOXe7cy7ROZdE8QHlZc65Bs65U5xzSd74bu8ANMA8IN27PsSb77zx\nYd7ZTMlAG+Bd4D2gjZklm1nQ+x7zKnCNIgA0qtWIpaOWcsEpFzBo5iBe/OxFvyOJhK3K+JzDOKCR\ntyVxM3AbgHPuU2Am8BmwELjBOVfkHZf4PbCI4rOhZnpzRSpcQo0EskZm0a5FO4bOHkrmJ5l+RxIJ\nS/qT3RKVdoZ20mdaH17/5nUm9J/AqPNH+R1JpNLpT3aLHEOdYB0WXLmAtOQ0Rr88mnHvj/M7kkhY\nUTlI1KoVV4tXhr9Cj9Y9uOaVa3jmvWf8jiQSNlQOEtVqxNbgpaEv0e/Mftyw4AbGvjPW70giYUHl\nIFEvPjaeWZfPYvDZg7lp0U08/ObDfkcS8Z3KQQQIBoJkDslk+LnDuXXJrdybe6/fkUR8Fet3AJFw\nERsTy+SBkwkGgtyZcyehohD3dLqH4j8NJhJdVA4iJQRiAozvP564mDjue/0+CosKeajLQyoIiToq\nB5HDxFgMz/V9jmAgyCNvPUKoKMSY7mNUEBJVVA4iZYixGJ7q9RTxsfGMeWcMoaIQT/V6ihjTYTqJ\nDioHkSMwMx7r9hjxgXgefPNBQkUhnuvzHIGYgN/RRCqdykHkKMyMBzo/QDAQ5J7l9xAqCjG+/3hi\nY/TSkepNz3CRYzAz7u50N8FAkL9m/5VQUYjJAycTF4jzO5pIpVE5iJTTXzr8hfjYeP6c9Wf2HdjH\n9MHTCQaCfscSqRQ6uiZyHP70yz/xeI/HmbNqDkNmDqFwf6HfkUQqhcpB5DjdeMmNPNv7WV754hX6\nZ/Znz749fkcSqXAqB5ET8F8p/8W4fuNY/NVi+kzvw67QLr8jiVQolYPICfrthb9l0sBJ5OTn0HNq\nT3YU7vA7kkiFUTmInIQR541g2qBpvLX+LbpP6c72vdv9jiRSIVQOIidp6LlDmXX5LPK+y6Pr5K5s\n3bPV70giJ03lIFIBBp49kDlD5/DRxo9Im5TGD7t/8DuSyElROYhUkD5n9GHesHl8/sPndJrYiY07\nN/odSeSEqRxEKlD31t159YpXWbt1LR0ndmTDjg1+RxI5ISoHkQqWlpzGwisXUvBjAakZqRT8WOB3\nJJHjpnIQqQS/bvVrFo9YzMZdG+kwoQP52/L9jiRyXFQOIpXk0tMuZemopWzdu5XUjFS+2vKV35FE\nyk3lIFKJUk5NITs9m12hXXTI6MDqH1b7HUmkXFQOIpXsglMuIGd0DvsP7Cc1I5XPNn/mdySRY1I5\niFSBc5ueS+7oXGIsho4ZHfnXxn/5HUnkqFQOIlXkrMZnkTs6l/jYeDpN7MT7G973O5LIEakcRKpQ\nm0ZtWD56OXWDdUmbmMaKghV+RxIpk8pBpIolN0hm+VXLaVyrMV0nd+XNb970O5JIKSoHER+0rN+S\n3NG5nFr3VLpP6U5Ofo7fkUQOUe5yMLOAmX1gZvO921PNbLWZfWJm480szhs3M3vCzNaY2b/M7KIS\nXyPdzL70Luklxi82s4+9xzxhZlaRixQJRy3qtSBndA6tElrRa2ovlqxd4nckkZ8cz5bDH4BVJW5P\nBc4Cfg7UBK7xxnsCbbzLdcA/AMysIXAXcAnQDrjLzBp4j/mHN/fg43qcwFpEIs4pdU4hJz2HNo3a\n0GdaHxZ8ucDvSCJAOcvBzBKB3sALB8eccwucB3gXSPTu6g9M8u56B0gws+ZAdyDLObfFObcVyAJ6\nePfVc8697X2tScCAilqgSLhrUrsJy0Yt45ym5zAgcwBzP5/rdySRcm85jAVuAQ4cfoe3O2kksNAb\nagGsLzGlwBs72nhBGeMiUaNRrUYsHbWUi5pfxJBZQ5j92Wy/I0mUO2Y5mFkfYJNzbuURpjwDLHfO\nvX7wIWXMcScwXlaW68wsz8zyNm/efIzkIpEloUYCi0cupn1ie4bNHsa0j6f5HUmiWHm2HC4D+plZ\nPpAJpJnZFAAzuwtoAtxcYn4BcFqJ24nAd8cYTyxjvBTn3PPOuRTnXEqTJk3KEV0kstSLr8drV75G\nh1YdGDFnBBM/nOh3JIlSxywH59ztzrlE51wSMAxY5pwbYWbXUHwcYbhzruTupnnAKO+spfbAdufc\nBmAR0M3MGngHorsBi7z7dphZe+8spVGAdrpK1KoTrMP8K+bT5fQuXDX3Kv658p9+R5IodDKfc3gW\naAa8bWYfmtmd3vgCYC2wBvgn8N8AzrktwL3Ae97lHm8M4HqKD3avAb4CXjuJXCIRr1ZcLeYNn0fP\nNj25bv51PP3u035HkihjxScIRZ6UlBSXl5fndwyRSlW4v5Chs4cyd/Vc/t7t79x06U1+R5IIZmYr\nnXMp5ZmrT0iLhLH42HhmXT6Ly9tezs2Lb+bBNx70O5JEiVi/A4jI0cUF4pg2eBpxgThuX3o7oaIQ\nd3S4A/0hAalMKgeRCBAbE8ukAZMIBoLclXMXhfsLuS/tPhWEVBqVg0iECMQEGNdvHMGYIA+88QCh\nohAPd31YBSGVQuUgEkFiLIZn+zxLMBDk0bcfJVQUYmyPsSoIqXAqB5EIY2Y80fMJ4mPjeeztxygs\nKuSZ3s8QYzq/RCqOykEkApkZj3R9hGAgyN/e+BuhohD/7PtPAjEBv6NJNaFyEIlQZsb9afcTH4jn\n/3L/j1BRiIwBGcTG6GUtJ0/PIpEIZmbc1fEugoEg/7vsf9l3YB9TBk4hLhDndzSJcCoHkWrg9l/f\nTnxsPP+z+H/YV7SPzCGZBANBv2NJBNMRLJFq4uZLb+bJnk/y0ucvMWjGIPbu3+t3JIlgKgeRauT3\n7X7Pc32e49UvX6V/Zn9279vtdySJUCoHkWrmuouvY3y/8WR9lUWfaX3YFdrldySJQCoHkWroqguv\nYvLAyeSuy6XH1B7sKNzhdySJMCoHkWrqyvOuJHNwJu8UvEO3Kd3Ytneb35EkgqgcRKqxy8+5nFmX\nz2LldyvpMqkLW/ZsOfaDRFA5iFR7A84awEtDX+KTTZ+QNjGNzbs2+x1JIoDKQSQK9D6jN/OGz2P1\nv1fTaWInNu7c6HckCXMqB5Eo0e1n3VhwxQK+3vY1HSd25Lsd3/kdScKYykEkinRK7sSiEYso+LGA\n1IxU1m9f73ckCVMqB5Eo86uWvyJrZBabd22mQ0YHvt76td+RJAypHESiUPvE9iwdtZTte7eTmpHK\nmi1r/I4kYUblIBKlLj71YrLTs9mzfw8dJnTg8x8+9zuShBGVg0gUO/+U88lJz+GAO0DHjI58sukT\nvyNJmFA5iES5c5qeQ+7oXAIxATpN7MRH33/kdyQJAyoHEeHMxmeSOzqXmrE16TSxE3nf5fkdSXym\nchARAFo3bM3yq5ZTv0Z9Ok/qzDsF7/gdSXykchCRnyQlJLF89HKa1m5K18ldeeObN/yOJD5ROYjI\nIU6rfxq5o3NpUbcF3ad0J/vrbL8jiQ9UDiJSyql1TyV3dC7JCcn0mtaLxV8t9juSVDGVg4iUqVmd\nZmSnZ3NmozPpO70vr37xqt+RpAqpHETkiJrUbsKy9GWc1+w8Bs4YyMufv+x3JKki5S4HMwuY2Qdm\nNt+7nWxmK8zsSzObYWZBbzzeu73Guz+pxNe43RtfbWbdS4z38MbWmNltFbc8ETlZDWs2ZMnIJVx8\n6sVcPutyZn06y+9IUgWOZ8vhD8CqErcfAsY459oAW4GrvfGrga3OudbAGG8eZtYWGAacA/QAnvEK\nJwA8DfQE2gLDvbkiEibq16jP4hGLaZ/YnmEvDmPqv6b6HUkqWbnKwcwSgd7AC95tA9KA2d6UicAA\n73p/7zbe/Z29+f2BTOdcoXPua2AN0M67rHHOrXXOhYBMb66IhJG68XVZeOVCUlulMvKlkUz4YILf\nkaQSlXfLYSxwC3DAu90I2Oac2+/dLgBaeNdbAOsBvPu3e/N/Gj/sMUcaF5EwUztYm/lXzKfrz7ry\n23m/5bm85/yOJJXkmOVgZn2ATc65lSWHy5jqjnHf8Y6XleU6M8szs7zNm/Xv4Ir4oVZcLeYOm0vv\nNr353au/48kVT/odSSpBebYcLgP6mVk+xbt80ijekkgws1hvTiJw8N8cLABOA/Durw9sKTl+2GOO\nNF6Kc+5551yKcy6lSZMm5YguIpWhRmwN5gydw8CzBnLjwht57K3H/I4kFeyY5eCcu905l+icS6L4\ngPIy59yVQDYwxJuWDsz1rs/zbuPdv8w557zxYd7ZTMlAG+Bd4D2gjXf2U9D7HvMqZHUiUmmCgSAz\nhszgN+f8hj9l/YkHXn/A70hSgWKPPeWIbgUyzew+4ANgnDc+DphsZmso3mIYBuCc+9TMZgKfAfuB\nG5xzRQBm9ntgERAAxjvnPj2JXCJSReICcUwdNJVgIMhflv2FUFGIu1LvovgcFIlkVvxLfeRJSUlx\neXn6s8Ii4aDoQBHXvnItEz6cwG2X3cYDnR9QQYQhM1vpnEspz9yT2XIQEQEgEBPghX4vEB+I58E3\nHyRUFOLRbo+qICKYykFEKkSMxfBM72cIBoL8/Z2/EyoK8XjPx4kx/ZWeSKRyEJEKY2aM7TGWYCDI\no28/SmFRIc/2eVYFEYFUDiJSocyMh7s+THxsPPe/fj+hohDj+o0jEBPwO5ocB5WDiFQ4M+O+tPuI\nD8RzZ86d7Duwj4kDJhIbo7ecSKGflIhUmjtS7yAYCHLb0tvYV7SPqYOmEheI8zuWlIPKQUQq1a2/\nupVgIMjNi28mVBRixpAZxMfG+x1LjkFHiUSk0t106U081fMp5q6ey6CZg9i7f6/fkeQYVA4iUiVu\naHcDz/d5nte+fI2+0/uye99uvyPJUagcRKTKXHvxtUzoP4FlXy+j97Te7Azt9DuSHIHKQUSqVPoF\n6UweOJnX171Ojyk9+LHwR78jSRlUDiJS5a74+RVkDslkxbcr6Da5G9v2bvM7khxG5SAivhjSdgiz\nL5/N+xvep/Okzvx797/9jiQlqBxExDf9z+rP3GFz+XTTp6RNSmPTrk1+RxKPykFEfNWzTU/mXzGf\nL//9JZ0mduL7nd/7HUlQOYhIGOhyehdeu/I11m1bR2pGKt/++K3fkaKeykFEwkJqUiqLRixiw44N\npGak8s32b/yOFNVUDiISNi5reRlZI7P4YfcPdJjQgbVb1/odKWqpHEQkrFySeAnL0pexI7SD1IxU\nvvz3l35HikoqBxEJOxc1v4js9Gz27t9LakYqqzav8jtS1FE5iEhYOq/ZeeSk53DAHaDjxI58sukT\nvyNFFZWDiIStc5qeQ+7oXGJjYumY0ZEPNnzgd6SooXIQkbB2ZuMzWT56ObWDtUmblMZ7377nd6So\noHIQkbD3s4Y/Y/no5TSo0YAuk7vw9vq3/Y5U7akcRCQitEpoxfKrltOsdjO6TenG8nXL/Y5Urakc\nRCRiJNZLJHd0Lon1Euk5tSdL1y71O1K1pXIQkYjSvG5zctJzOL3B6fSZ3odFaxb5HalaUjmISMRp\nVqcZ2enZnNX4LPpl9uOV1a/4HanaUTmISERqXKsxy0Yt4/xm5zNo5iDmrJrjd6RqReUgIhGrQc0G\nZI3M4hen/oLfzPoNMz6Z4XekakPlICIRrX6N+iwasYjLWl7GFXOuYPJHk/2OVC2oHEQk4tWNr8uC\nKxbQMakj6S+nM/6D8X5HinjHLAczq2Fm75rZR2b2qZnd7Y13NrP3zexDM3vDzFp74/FmNsPM1pjZ\nCjNLKvG1bvfGV5tZ9xLjPbyxNWZ2W8UvU0Squ9rB2swfPp/urbtz9byreTbvWb8jRbTybDkUAmnO\nufOBC4AeZtYe+AdwpXPuAmAa8Fdv/tXAVudca2AM8BCAmbUFhgHnAD2AZ8wsYGYB4GmgJ9AWGO7N\nFRE5LjXjavLy0Jfpe0Zfrn/1ep5Y8YTfkSLWMcvBFdvp3YzzLs671PPG6wPfedf7AxO967OBzmZm\n3nimc67QOfc1sAZo513WOOfWOudCQKY3V0TkuMXHxjP7N7MZdPYg/rDwDzzy5iN+R4pI5Trm4P2G\n/yGwCchyzq0ArgEWmFkBMBJ40JveAlgP4JzbD2wHGpUc9xR4Y0caLyvHdWaWZ2Z5mzdvLt8KRSTq\nBANBMgdnMuzcYdyy5BbuW36f35EiTrnKwTlX5O0+SgTamdm5wE1AL+dcIjAB+Ls33cr6EicwXlaO\n551zKc65lCZNmpQnuohEqbhAHFMGTmHkeSO5I/sO7sy+E+fKfGuRMsQez2Tn3DYzy6H4+MD53hYE\nwAxgoXe9ADgNKDCzWIp3OW0pMX5QIv/ZFXWkcRGRExaICTCh/wSCgSD3Lr+XUFGIv3X+G8V7uuVo\nynO2UhMzS/Cu1wS6AKuA+mZ2hjetqzcGMA9I964PAZa54rqeBwzzzmZKBtoA7wLvAW3MLNnMghQf\ntJ5XIasTkagXiAnwfN/nuT7leh568yFuXnSztiDKoTxbDs2Bid5ZRTHATOfcfDO7FnjRzA4AW4Hf\nevPHAZPNbA3FWwzDAJxzn5rZTOAzYD9wg3OuCMDMfg8sAgLAeOfcpxW2QhGJejEWw9O9niYYCDJ2\nxVhCRSGe7PUkMaaPeh2JRWqDpqSkuLy8PL9jiEgEcc5x25LbePith7nmwmt4ru9zUVUQZrbSOZdS\nnrnHdcxBRCSSmRkPdnmQ+Nj44mMQB0KM7zeeQEzA72hhR+UgIlHFzLin0z3ExcRxZ86dhIpCTB44\nmdgYvR2WpP8bIhKV7ki9g/jYeG5dciv7ivYxbfA0goGg37HCRvTsbBMROcwtl93CmO5jeHHViwyZ\nOYTC/YV+RwobKgcRiWp/bP9Hnun1DK988QoDZgxgz749fkcKCyoHEYl61//iel7o+wKL1iyi7/S+\n7N632+9IvlM5iIgAV190NRkDMsjOz6bX1F7sDO089oOqMZWDiIhn1PmjmDpoKm988wbdp3Rn+97t\nfkfyjcpBRKSEYecOY8aQGbz77bt0ndyVrXu2+h3JFyoHEZHDDG47mDm/mcNHGz+i86TO/LD7B78j\nVTmVg4hIGfqe2Ze5w+by2ebPSJuYxqZdm/yOVKVUDiIiR9CjdQ9eveJV1mxZQ8eMjmzYscHvSFVG\n5SAichSdT+/Ma1e+xjfbvyE1I5WCHwv8jlQlVA4iIseQmpTK4pGL2bhrI6kZqazbts7vSJVO5SAi\nUg6/PO2XZI3MYsueLXTI6MBXW77yO1KlUjmIiJRTuxbtWDpqKTtDO0nNSOWLf3/hd6RKo3IQETkO\nFzW/iJz0HEJFIVIzUvls82d+R6oUKgcRkeP082Y/J2d0DgAdMzryr43/8jdQJVA5iIicgLZN2pI7\nOpdgIEiniZ14f8P7fkeqUCoHEZETdEajM1h+1XLqBOvQeVJn3v32Xb8jVRiVg4jISTi9weksH72c\nhjUb0mVSF9785k2/I1UIlYOIyElqldCK3NG5NK/bnO5TupObn+t3pJOmchARqQCJ9RLJSc+hVUIr\nek7tyZK1S/yOdFJUDiIiFaR53eZkp2fTumFr+kzrw2tfvuZ3pBOmchARqUBNazclOz2btk3aMmDG\nAOatnud3pBOichARqWCNajVi6ailXHDKBQyeOZgXP3vR70jHTeUgIlIJGtRsQNbILNq1aMfQ2UOZ\n/vF0vyMdF5WDiEglqRdfj0UjFvGrlr9ixEsjmPjhRL8jlZvKQUSkEtUJ1mHBlQtIS07jqrlX8cL7\nL/gdqVxUDiIilaxWXC3mDZtH99bdufaVa3nmvWf8jnRMKgcRkSpQM64mLw99mX5n9uOGBTcw9p2x\nfkc6KpWDiEgViY+NZ9blsxh89mBuWnQTD73xkN+RjuiY5WBmNczsXTP7yMw+NbO7vXEzs/vN7Asz\nW2VmN5YYf8LM1pjZv8zsohJfK93MvvQu6SXGLzazj73HPGFmVhmLFRHxWzAQJHNIJsPPHc5tS2/j\n3tx7/Y5UpthyzCkE0pxzO80sDnjDzF4DzgZOA85yzh0ws6be/J5AG+9yCfAP4BIzawjcBaQADlhp\nZvOcc1u9OdcB7wALgB5A5H60UETkKGJjYpk8cDJxgTjuzLmTwqJC7u10L+H0e/Exy8E554Cd3s04\n7+KA64ErnHMHvHmbvDn9gUne494xswQzaw50BLKcc1sAzCwL6GFmOUA959zb3vgkYAAqBxGpxgIx\nASb0n0AwJsj9r99PqCjEQ10eCpuCKM+WA2YWAFYCrYGnnXMrzOxnwFAzGwhsBm50zn0JtADWl3h4\ngTd2tPGCMsbLynEdxVsYtGzZsjzRRUTCVozF8Fzf5wgGgjzy1iOEikKM6T4mLAqiXOXgnCsCLjCz\nBOAlMzsXiAf2OudSzGwQMB74NVDWqtwJjJeV43ngeYCUlJQy54iIRJIYi+GpXk8RHxvPmHfGULi/\nkKd7P02M+Xu+ULnK4SDn3DZ1mw1iAAAGTElEQVRvN1APin/DP/gHQ14CJnjXCyg+FnFQIvCdN97x\nsPEcbzyxjPkiIlHBzHis22MEA0EeevMhQkUhnu/7PIGYgG+ZynO2UhNviwEzqwl0AT4HXgbSvGmp\nwBfe9XnAKO+spfbAdufcBmAR0M3MGphZA6AbsMi7b4eZtffOUhoFzK24JYqIhD8z42+d/8adHe5k\n/IfjGT13NPsP7PctT3m2HJoDE73jDjHATOfcfDN7A5hqZjdRfMD6Gm/+AqAXsAbYDVwF4JzbYmb3\nAu958+45eHCa4oPbGUBNig9E62C0iEQdM+PuTncTDAT5a/Zf2Ve076ezmqo8S/FJRZEnJSXF5eXl\n+R1DRKRSPPrWo/w5688MPGsgmUMyCQaCJ/01zWylcy6lPHP1CWkRkTD0p1/+icd7PM5Ln7/E4JmD\n2bt/b5V+f5WDiEiYuvGSG/lH738w/4v59M/sz559e6rse6scRETC2O9Sfse4fuPI+iqLPtP7sCu0\nq0q+r8pBRCTM/fbC3zJp4CRy8nPoObVnlRTEcX3OQURE/DHivBHExcSRtTaLmnE1K/376WwlEZEo\nobOVRETkpKgcRESkFJWDiIiUonIQEZFSVA4iIlKKykFEREpROYiISCkqBxERKSViPwRnZpuBdSf4\n8MbADxUYx0/VZS3VZR2gtYSj6rIOOLm1tHLONSnPxIgth5NhZnnl/ZRguKsua6ku6wCtJRxVl3VA\n1a1Fu5VERKQUlYOIiJQSreXwvN8BKlB1WUt1WQdoLeGouqwDqmgtUXnMQUREji5atxxEROQoqnU5\nmNlpZpZtZqvM7FMz+4M33tDMsszsS++/DfzOeixmVsPM3jWzj7y13O2NJ5vZCm8tM8ws6HfW8jCz\ngJl9YGbzvduRuo58M/vYzD40szxvLOKeXwBmlmBms83sc+81c2kkrsXMzvR+HgcvP5rZHyNxLQBm\ndpP3mv/EzKZ77wWV/nqp1uUA7Af+xzl3NtAeuMHM2gK3AUudc22Apd7tcFcIpDnnzgcuAHqYWXvg\nIWCMt5atwNU+ZjwefwBWlbgdqesA6OScu6DE6YWR+PwCeBxY6Jw7Czif4p9PxK3FObfa+3lcAFwM\n7AZeIgLXYmYtgBuBFOfcuUAAGEZVvF6cc1FzAeYCXYHVQHNvrDmw2u9sx7mOWsD7wCUUfxgm1hu/\nFFjkd75y5E+k+MWZBswHLBLX4WXNBxofNhZxzy+gHvA13nHISF7LYfm7AW9G6lqAFsB6oCHF/6zz\nfKB7VbxeqvuWw0/MLAm4EFgBNHPObQDw/tvUv2Tl5+2K+RDYBGQBXwHbnHP7vSkFFD+Zwt1Y4Bbg\ngHe7EZG5DgAHLDazlWZ2nTcWic+v04HNwARvd98LZlabyFxLScOA6d71iFuLc+5b4FHgG2ADsB1Y\nSRW8XqKiHMysDvAi8Efn3I9+5zlRzrkiV7ypnAi0A84ua1rVpjo+ZtYH2OScW1lyuIypYb2OEi5z\nzl0E9KR4t2UHvwOdoFjgIuAfzrkLgV1EwG6Xo/H2w/cDZvmd5UR5x0X6A8nAqUBtip9rh6vw10u1\nLwczi6O4GKY65+Z4wxvNrLl3f3OKfxOPGM65bUAOxcdREsws1rsrEfjOr1zldBnQz8zygUyKdy2N\nJfLWAYBz7jvvv5so3q/djsh8fhUABc65Fd7t2RSXRSSu5aCewPvOuY3e7UhcSxfga+fcZufcPmAO\n8Euq4PVSrcvBzAwYB6xyzv29xF3zgHTvejrFxyLCmpk1MbME73pNip80q4BsYIg3LezX4py73TmX\n6JxLoniTf5lz7koibB0AZlbbzOoevE7x/u1PiMDnl3Pue2C9mZ3pDXUGPiMC11LCcP6zSwkicy3f\nAO3NrJb3fnbw51Lpr5dq/SE4M/sV8DrwMf/Zv/2/FB93mAm0pPh//uXOuS2+hCwnMzsPmEjx2Qox\nwEzn3D1mdjrFv4E3BD4ARjjnCv1LWn5m1hH4k3OuTySuw8v8knczFpjmnLvfzBoRYc8vADO7AHgB\nCAJrgavwnmtE3lpqUXwg93Tn3HZvLFJ/LncDQyk++/ID4BqKjzFU6uulWpeDiIicmGq9W0lERE6M\nykFEREpROYiISCkqBxERKUXlICIipagcRESkFJWDiIiUonIQEZFS/h8Bk5ndqThmxQAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10990d8d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 绘制不同聚类数目下的模型得分\n",
    "plt.plot(Ks, CH_scores, 'g-')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同样在 K = 40 时得分最高. 因此下面采用 K = 40 训练模型进行预测."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 用最佳的 K 训练, 保存预测结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 采用 K = 40 重新训练模型, 并进行预测\n",
    "n_clusters = 40\n",
    "mb_kmeans = MiniBatchKMeans(n_clusters=n_clusters, batch_size=1000)\n",
    "mb_kmeans.fit(df_FE)\n",
    "label = mb_kmeans.predict(df_FE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 获取 user_id\n",
    "user_id = pd.read_csv('users.csv')['user_id']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 整合结果\n",
    "category = pd.DataFrame(label, columns=['category'])\n",
    "result = pd.concat([user_id, category], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 保存结果文件\n",
    "result.to_csv('UserClassification.csv')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
